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The invention further provides for a method of verifying the integrity or origin of a 
program comprising: 

embedding a watermark W in the state of a program as the program is being run 
5 with a particular input sequence /; 

building a recognizer R concurrently with the input / and watermark W wherein the 
recognizer is adapted to extract the watermark graph from other dynamically 
allocated data structures wherein R is kept separately from the program; wherein R 
is adapted to check for a number n. /?, in a preferred embodiment, being the product 
10 of two primes and wherein n is embedded in the topology of W, 

Other properties of 1/1/ may be used to compute the signature. 

The number n may be derived from any combination of numbers depending on the 
15 context and application. 

Preferably the program or code is further adapted to be resistant to tampering, 
preferably by means of obfuscation or by adding tamper-proofing code. 

20 Preferably the watermarks W are chosen from a class of graphs G wherein each 
member of G has one or more properties, such as planarity, said property being 
capable of being tested by integrity-testing software. 

In an alternative embodiment, the watenmark may be rendered tamperproof to 
25 certain transformations, such as attacks, by expanding each node of the watermark 
graph into a y-cycle, where j may be any number, in a preferred embodiment, a small 
number from 1 to 5. 

In a broad aspect, the recognizer R checks for the effect of the watermarking code 
30 on the execution state of the application thereby preserving the ability to recognize 
the watermark in cases where semantics-preserving transformations have been 
applied to the application. 



35 



In a further aspect, the invention provides for a method of watenmarking software 
including the steps of: 
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8. A method as claimed in any preceding claim further comprising building a 
recognizer R concurrently with the input / and watermark W. 

9. A method as claimed in claim 8 wherein R is a function adapted to identify and 
extract the watermark graph from all other dynamically allocated data 
structures. 

10. A method as claimed in either claim 8 or 9 wherein the watemiark W 
incorporates a marker that will allow R to recognize it easily. 

11. A method as claimed in any one of claims 8 to 10 wherein R is retained 
separately from the program and whereby R inspects the state of the program. 

12. A method as claimed in any one of claims 8 to 11 wherein R is dynamically 
linked with the program when it Is checked for the existence of a watermark. 

13. A method as claimed in any preceding claim wherein the application of which 
the object fomis a part is obfuscated or incorporates tamper-proofing code. 

14. A method as claimed in any one of claims 8 to 12 wherein R checks W^for a 
signature property s(W). 

15. A method as claimed in claim 14 including the creation of a number n which 
may be embedded in the topology of whereby the signature property may 
be evaluated by testing one or more numeric properties of n. 

16. A method as claimed in claim 15 wherein the signature property is evaluated 
by testing whether n is the product of two primes. 

17. A method of verifying the integrity or origin of a program comprising: 
embedding a watermark W in the state of a program as the program is being 
run with a particular input sequence /; 

building a recognizer R concurrently with the input / and watermark W wherein 
the recognizer is adapted to extract the watermark graph from other dynamic 
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staictures on the heap or stack wherein R is kept separately from the program; 
wherein R is adapted to check for a number n, n, in a preferred embodiment, 
being the product of two primes and wherein n is embedded in the topology of 

W. 



18. A method as claimed in claim 17 where other properties of s(W) are used to 
compute the signature. 

19. A method as claimed in either claim 17 or 18 wherein the number n is derived 
from any combination of numbers depending on the context and application. 

20. A method as claimed in any one of claims 17 to 19 wherein the program or 
code is further adapted to be resistant to tampering, preferably by means of 
obfuscation or by adding tamper-proofing code. 

21. A method as claimed in any one of claims 17 to 20 wherein the recognizer R 
checks for the effect of the watenmarking code on the execution state of the 
application thereby preserving the ability to recognize the watermark in cases 
where semantics-preserving transformations have been applied to the 
application. 

22. A method of watermarking software including the steps of: 
embedding a watermark in a static string; and 

applying an obfuscation technique whereby this static string is converted into 
executable code. 

23. A method of fingerprinting software wherein a plurality of watermarked 
programs obtained as claimed in any preceding claim are produced. 

24. A method of fingerprinting software as claimed in claim 23 wherein the 
watermarked programs each of which has a number n with a common prime 
factor p. 
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25. 

5 

26, 

10 27. 

28. 
15 29. 
30. 
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A method of watermarking software wherein the watermark W is chosen from 
a class of graphs G wherein each member of G has one or more properties, 
such as planarity, said property being capable of being tested by integrity- 
testing software. 

A method of watermarking software as claimed in claim 25 wherein the 
watermark may rendered tamperproof to certain transformations by 
subjecting the watermark graph to one or more local transformations. 

A method of watermarking software as claimed in claim 26 wherein each 
node of the watermark graph is expanded into a cycle. 

A method substantially as herein described with reference to the drawings. 

Software written to perform the method as claimed in any preceding claim. 

A computer programmed to perform the method as claimed in any one of 
claims 1 to 27. 
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Claims 1 to 22 are linked by the common inventive subject matter of 
"watermarking a program or software object with a watermark wherein the 
watermark is stored in the state of a program as the program is being run with a 
particular input sequence." 

Claim 23 does not have this feature. Instead it has the features of embedding a 
watermark in a static string and obfuscating the static string. 

Claim 24 is defines a different invention again from the previous two sets of 
claims. It defines a watermarking method wherein the watermark is chosen from a 
class of graphs and the properties are capable of being tested by integrity testing 
software. 



Consequently, the following parts of the international application were the subject of international preliminary 
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The invention further provides for a method of verifying the integrity or origin of a 
program including: 

watermarking the program with a watermark W in the state of a program as the 
program is being run with a particular input sequence /; 

building a recognizer R concurrently with the input / and watermark W wherein the 
recognizer is adapted to extract the watermark graph from other dynamically 
allocated data structures wherein R is kept separately from the program; wherein R 
is adapted to check for a number n, n, in a preferred embodiment, being the product 
of two primes and wherein n is embedded in the topology of W, 

Preferably, the signature property may be evaluated by testing for a specific result 
from a hard computational problem. 



The number n may be derived from any combination of numbers depending on the 
15 context and application. 

Preferably the program or code is further adapted to be resistant to tampering, 
preferably by means of obfuscation or by adding tamper-proofing code. 

20 Preferably the watermarks W are chosen from a class of graphs G wherein each 
member of G has one or more properties, such as planarity, said property being 
capable of being tested by integrity-testing software. 

in an altemative embodiment, the watermark may be rendered tamperproof to 
25 certain transformations, such as attacks, by expanding each node of the watermark 
graph into a /-cycle, where j may be any number from 1 to 5. 

In a broad aspect, the recognizer R checks for the effect of the watermarking code 
on the execution state of the application thereby preserving the ability to recognize 
30 the watermark in cases where semantics-preserving transformations have been 
applied to the application. 

In a further aspect, the invention provides for a method of watermarking software 
including the steps of: 

- . 03125722 




PCT/NZ99/00081 
Received 10 July 2000 



5 9. 
10. 

10 

11. 

15 12. 
13. 

20 

14. 
15. 

25 

16. 

30 

17- 



A method as claimed in any preceding claim further comprising building a 
recognizer R concun-ently with the input / and watermark W, 

A method as claimed in claim 8 wherein R is a function adapted to identify and 
extract the watermark graph from ail other dynamic structures on the heap or 
stack. 

A method as claimed in either claim 8 or 9 wherein the watermark W 
incorporates a marker that will allow R to recognize it easily. 

A method as claimed in any one of claims 8 to 10 wherein R is retained 
separately from the program and whereby R inspects the state of the program. 

A method as claimed in any one of claims 8 to 11 wherein R is dynamically 
linked with the program when it is checked for the existence of a watermark. 

A method as claimed in any preceding claim wherein the application of which 
the object forms a part is obfuscated or incorporates tamper-proofing code. 

A method as claimed in any one of claims 8 to 12 wherein R checks IV for a 
signature property s(W). 

A method as claimed in claim 14 wherein the signature property s(WO is 
evaluated by testing for a specific result from a hard computational problem. 

A method as claimed in either claim 14 or claim 15 including the creation of a 
number n which may be embedded in the topology of W, wh^ereby the 
signature property may be evaluated by testing one or more numeric properties 

of n. 

A method as claimed in claim 16 wherein the signature property is evaluated 
by testing whether n is the product of two primes. 
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18. A method of verifying the integrity or origin of a program including: 

watermarking the program with a watermark M/, wherein the watermark W is 
stored in the state of a program as the program is being run with a particular 
input sequence /; 

5 building a recognizer R concurrently with the input / and watermark W wherein 

the recognizer is adapted to extract the watenmaric graph from other 
dynamically allocated data wherein R is kept separately from the program; 
wherein R is adapted to check for a number n. 

10 19. A method of verifying the integrity of origin of a program as claimed in claim 1 8, 
wherein n is the product of two primes and wherein n is embedded in the 
topology of W. 

20. A method as claimed in either claim 18 or 19 wherein the number n is derived 
15 from any combination of numbers depending on the context and application. 

21. A method as claimed in any one of claims 18 to 20 wherein the program or 
code is further adapted to be resistant to tampering, preferably by means of 
obfuscation or by adding tamper-proofing code. 

20 

22. A method as claimed in any one of claims 18 to 21 wherein the recognizer R 
checks for the effect of the watermarking code on the execution state of the 
application thereby preserving the ability to recognize the watermark in cases 
where semantics-preserving transformations have been applied to the 

25 application. 

23. A method of watermarking software including the steps of: 
embedding a watermark in a static string; and 

applying an obfuscation technique whereby this static string is converted into 
30 executable code. 

24. A method of watermarking software wherein the watermark IV is chosen from a 
class of graphs G wherein each member of G has one or more properties, 
such as planarity, said property being capable of being tested by integrity- 

35 testing software. 
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A method of watermarking software as claimed in daim 24 wherein the 
watenmark is rendered tamperproof to certain transfomiations by subjecting the 
watermark graph to one or more local transformations. 

A method of watermarking software as claimed in daim 25 wherein each node 
of the watermark graph is expanded into a cycle. 

A method of fingerprinting software wherein a plurality of watermarked 
programs obtained as claimed in any preceding claim are produced. 

A method of fingerprinting software as claimed in claim 27 wherein the 
watermarked programs each of which has a number n with a common prime 
factor p. 

A method substantially as herein described with reference to the drawings. 

Software written to perform the method as claimed in any preceding claim. 

A computer programmed to perform the method as daimed in any one of 
claims 1 to 27. 
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WATERMARKING OF DIGITAL OBJECT 

FIELD OF THE INVENTION 

The present invention is related to techniques for watermaridng digital data, 
and more specifically, to watermarking digital data such as images and audio data for 
authenticating copyright ownership. 

BACKGROUND 

Because of the rapid increase of electronic commerce in the recent years, 
secured data transaction is becoming more and more important To prevent 
electronic data to be appropriated by unauthorized parties, cryptographic methods 
have been used to transmit digital data between consenting parties to guard against 
imintended e;q)0sure to other parties. Various kinds of data, including military 
information, financial transaction, personal data, and the like can be transmitted via 
cryptography to protect the data. 

In the arena of protecting the rig^t to original art or literary worics, in the past, 
most violations were by unauthorized parties making physical copies of authorized 
copies of the original (e.g., copyrighted) works. Copies of physical artistic material, 
such as copyrighted paintings, photographs, phonogrq)hs, and analog audio tapes, are 
\isually perceptibly inferior to the originals. The degradation of fidelity in the 

1 
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copying process, e.g., in photocopying or photography, is a factor in deterring 
unauthorized copying of such material. Today, many visual, audio, literary, or other 
proprietary works are stored and transmitted digitally. Such digital material can be 
copied over and over without significant loss in fidelity. The risk to the owner of an 
original artistic work, or proprietary work, is that once the digital data are transmitted, 
if data suspected to be copies of the transmitted data are found, the verification of 
whether the suspect data are copied firom the originally transmitted data, for example, 
digital data of a piece of art work, is usually impossible. 

Recently, digital watermarking has been devised as a security technique to 
facilitate the identification of the source of digital material for the purpose of, for 
example, copyright enforcement. The watermark is an identification code that is 
imbedded in the original digital data and is preferably imperceptible to the human 
observer of the artistic work. One example of a scheme for watermarking involves 
inserting an identification string into a digital audio signal to substitute the 
insignificant bits of randomly selected audio samples with the bits of an identification 
code . Another example of watermarking relating to watermarking video digital 
works involves assigning a predetermined value to a predetermined coding parameter 
that, when modified, requires a plurality of fiirther parameters to be modified in order 
to correctly decode the video signal. In one watermaricing technique each copy of an 
object is mariced with an identifier code. More recently, a watermarking scheme in 
which a two-dimensional spread spectrum signal is added to an image has been 
proposed. To verify the watermark in a given image, the original image is subtracted 
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from the given image and the correlation of the difference image to the watermark 
signal is computed. 

Although much advance has been made in watermarking digital data, 
generally, prior techniques of watermarking suffer from a variety of shortcomings. 
Often, the original image is necessary to verify the presence of the watermark. 
Manipulative operations such as cropping to cut out a portion of the work and scaling 
to obtain a woric of larger or smaller size pose a considerable problem on the 
verification process. Many times the watermarking scheme is invertible, i.e., an 
attacker (or temperer), based on a first watermarked image but without knowledge of 
what the first watermark is, would be able to compute a second image and a second 
watermark such that inserting the second watermark into the second image would 
result in the first watermarked image. Such invertible watermarking schemes can 
make the verification of authentic copies of a copyrigjited work difficult Further, it 
may even lead to ownership disputes on valuable digital data since an unscrupulous 
person might appropriate another's watermarked material, subtract &om it his own 
watermark and claim the resulting product to be his own. Often, in prior 
watermarking techniques, changes in contrast or brightness may fool the verification 
algorithms, making them less reliable. Furthermore, many of the watermark schemes 
do not offer a mechanism for creating and managing watermarics. This means that if 
the same watermark is used to protect several works, compromise in the secrecy of 
the watermark will compromise the protection for all of the works. What is needed a 
watermarking technique that can these deficiencies. The present invention provides a 
watermarking technique that is resistant to cropping, invertible, resistant to brightness 
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or contrast changes, and will not compromise other related watermariced works if the 
mechanism of watermarking in one watermarked work is disclosed. 

SUMMARY 

The present invention provides a technique for identifying digital object using 
a digital watermark. This technique can be easily implemented using computers. 
The technique includes encrypting data derived from a set of source data on the 
digital object, deriving from the encrypted data a watermark, and incorporating the 
watermark into the source data. Preferably, the source data of the digital object are 
processed through a hash function to obtain a message digest (M) on the digital object 
and the message digest (M) is encrypted with a signature encryption key to obtain an 
encrypted message digest (S). Further, the encryption of the message digest (M) is 
preferably done with a public key-private key encryption system. Because the 
preferred mode is to process the source data through a one-way hash fimction to 
obtain a message digest, for clarity and convenience, the set of data for encryption 
derived from the set of source data is termed a "message" or "message digest" herein 
althou^ they do not necessarily have to have been through a one-way hash function. 
The set of encrypted data is called "encrypted message digest" herein for the same 
reason. 

The present watermaridng technique is versatile and can be advantageously 
employed to watermark a variety of digital objects, including audio, video, image, 
multimedia data, and the like. Further, the present technique offers high security 
because it is not easily foiled by attackers. For example, by a using a hash fimction in 
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conjunction with encryption in which a private key is kept confidential, the present 
technique offers the advantage of being not^invertibility, unlike many prior 
watermarking techniques. Therefore, it is very difficult for an attacker to 
backcompute the original watermark by the disclosure of inunediate information. To 
further increase security, in an embodiment of the present invention a public key 
encryption system is employed. As a result, a suspect object (i.e., an object suspected 
to be copied from an original, e.g., copyrighted, object) can be checked with the 
public key of the original object's owner to determine whether the watermaric is 
present without compromising the other watermarked objects. In the embodiment in 
which an encryption technique involving a public key is used, ownership of an object 
can be established to a neutral party, such as a court of law, using only the public key 
of the object's owner, without requiring the owner to reveal his private key. Also, the 
watermark is not removed by lossy compression (in which less perceptually 
important information is severed to reduce the size of the data file) or cropping (in 
which a portion of the object is cut). In the embodiment in which the watermaricing 
vector is orthogonal to the vector of the pixels into which the watermark is to be 
inserted, changes in the brightness or contrast will not fool the verification algorithm. 



BRIEF DESCRIPTION OF THE DRAWINGS 



The following figures are included to better illustrate the embodiments of the 
technique of the present invention. In these figures, like numerals represent like 
features in the several views. 
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Figure 1 is a block diagram showing an embodiment of the watermarking 
technique of the present invention. 

Figure 2 is a block diagram showing in further detail an embodiment of the 
watermaricing technique of Fig. 1 

Figure 3 shows a flow diagram of an embodiment of a watermarked object 
being obtained from a digital object according to the present invention. 

Figure 4 shows how a suspect object is evaluated to determine whether it is 
derived from a watermarked object, according to an embodiment technique of the 
present invention. 

Figure 5 shows a picture printed from a digital image. 

Figure 6 shows a picture printed from a digital image having a watermark 
incorporated into the digital image of Fig. S. 

Fig. 7 shows a correlation spread of Fig. 6. 

Fig. 8 shows a cropped and JPEG- compressed watermarked image of Fig. 5, 
showing the resistance to distortion by compression and cropping. 

Fig. 9 shows a correlation spread of Fig. 8, showing the sensitivity of the 
evaluation for the presence of watermaric in the object even when the object has been 
cropped and compressed. 

Fig. 10 shows a truncated watermarked image of Fig. 5, showing the resistance 
to distortion by truncation. 

Fig. 1 1 shows a correlation spread of Fig. 10, showing the sensitivity of the 
evaluation for the presence of watermark in the object even when the object has been 
truncated. " 
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DETAILED DESCRIPTION 

The present invention provides a technique for inserting a digital watermark in 
to a digital object and for evaluating a digital object for the watermark. An 
encryption technique is used in the derivation of the watermark and the watermark is 
incorporated into the digital object in such a way to render it extremely hard for an 
attacker (i.e., one who tampers with the watermarked data to remove or change the 
watermaric) to derive the original digital object from the watermarked digital object. 
The digital watermark is resistant to cropping, scaling, inadvertent distortions, as well 
as the intentional removal or corruption of the watermark by an attacker. 

A digital object that can be watermarked with the technique of the present 
invention is referred to as a "physical object" in that it can be an digital visual image; 
digital audio program, e.g., music; digital tactile data, e.g., data which can be 
transformed into tactile information that can be sensed by touch; multimedia data; or 
simply digital data strings that can be divided into discrete segments arrangeable into 
rows and columns of "pixels." One obvious example of such pixels would be, for an 
image, the pixels of digital information obtained by scanning a color photograph into 
digital information representing rows and columns of color dots (commonly known in 
visual display technology as "pixels"). Thus, a digitized photograph may contain 
hundreds of rows and columns of pixels. However, in this disclosure, "pixels" can be 
the discrete segments of digital data for other types of digital information, e.g., those 
mentioned *in the above. In the case of sound recording, for example, a sound signal 
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may be sampled by an A/D converter, which would output samples having values 
representative of the characteristics of the spund signal at particular time segments. 
The technique for obtaining pixels of data on visual images, audio signals, and other 
data streams are well known in the art. Any conventional technique for obtaining the 
digital objects is applicable. As anodier example, a digital camera or a computer 
running a computer graphics software can be used to generate digital images directly. 
Similarly, music can be converted from sound waves to digital data by using A/D 
converters. These digital objects, as well as other types of digital objects, are 
applicable in the present invention. 

An embodiment of the watermarking technique of the present invention is 
depicted in Fig. 1. A digital object, I(iJ), having m rows and n columns of pixels, is 
processed through a hash function 100 (e.g., a one-way hash function such as the 
MDS function) to result in a message digest M. The function of the hash function is 
to take the input data I(i J) and convert them into a fixed-size string (hash), and 
preferably a much shorter string for a large object, much as a fingerprint, of the input 
data I (i J). Preferably, the hash is generated in such a way that it is very difficult to 
generate the original input data firom the hash, i.e., the message digest M. This 
difficulty of back calculation is beneficial because if an attacker were able to derive 
the input data from die hash, he might be able to remove or change the watermark. 
Many hash functions are known in the art and can be q)plied in the present technique. 
One applicable one-way hash function (or message digest algorithm) known in the art 
is the MDS method. Other examples of applicable one-way hash functions include 
the SNEFRU function, SHA fimction, and HAVAL function. (See Schneier, B., 

8 
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Applied Cryptography^ John Wiley and Sons, 1993, pp. 333-346 for a discussion on 
one-way hash fiinctions,) A person skilled in the art will know how to apply such 
hash functions to a digital object In general, in selecting a hash fimction H(F) that 
operates on an arbitrary-length message F to obtain a fixed-length hash value, h. 



h = H(M), 



the following characteristics are desired: Given M, it is easy to compute h; given h, it 
is hard to compute M; and given M, it is hard to find another message M' with the 
property H(M) = H(M'). Preferably, the hash function is selected such that it is 
resistant to brute force attacks, as well as Birthday attacks, which are based on the 
occurrence of two random messages that return the same value throu^ the same hash 
fimction. 

The message digest M is then encrypted with a signature. The encryption 
algorithm can either be a public key-private key system (asymmetric data encryption 
algorithm) or a private key system (symmetric data encryption algorithm). Such an 
encryption imparts a imique feature in the message digest M that will distinguish 
these encrypted data from other data, thus acting much like a physical signature on a 
physical document If a private key alone is used for encryption, this private key 
preserves the secrecy of the original message and encrypted message digest is 
invertible, i.e., an attacker cannot back calculate the message digest M or the original 
object from the encrypted message digest However, if it is necessary to prove 
ownership; e.g., to a third party such as a court of law, to verify the owner's 
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signature, the owner would have to reveal his private key. With the private key 
revealed, his other watermarked objects that are marked with the same or similar 
private key watermark might be compromised since an attacker might be able to gain 
access to his private key to use it to obtain information on the watermarking of the 
other objects. 

A more preferable way would be to use a public key-private key encryption 
system (referred to as the "public key system" hereinafter unless specified otherwise). 
In encryption using the public key system, the user possess a matched pair of keys: a 
private key and a public key. The private key is kept secret and is known only to the 
user whereas the public key can be distributed widely. A message encrypted with 
either key can only be decrypted with the other key. If a user encrypts a message 
with his private key, then the message can only be decrypted with his public key. 
Because only the user has the private key, the encrypted message can be decrypted 
only with his public key. Smce the private key is known only to the encryptor, it is 
established that if the encrypted message can be decrypted with the user's public key, 
he must have encrypted the message, i.e., he has "signed" the message with his 
"signature." The strength of the signature is dependent on knowing that the public 
key of the user is genuine. For this reason, public keys are preferably notarized or 
certified by third parties. In this invention, when the public key system is used, the 
private key is used to encrypted the message to create the watermaric from the 
original image. When it is necessary to prove ownership of a watermaric, the public 
key can be provided, e.g., to a verifying third party, to verify the owner's signature. 
The advantage of using public key encryption in this invention is that since the 

10 
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private key is used to produce the watermark from the original object, it will be 
impossible, or extremely difficult, for an attacker to remove the watermark from the 
watermarked object to create a false original without the private key, even if he had 
access to the public key. 

In either the private key system case or the public key system case, the private 
key of the encryption algorithm is necessary to back-calculate from the encrypted M 
to obtain the original M. Many encryption algorithms are known in the art and can 
be used for this purpose. A good example is RSA algorithm. Other ^plicable public 
key encryption algorithms include ELGAMAL algorithm and DSS (digital signature 
standard) algorithm. Many other encryption algorithms are also known in the art, e.g., 
POHLIG-HELLMAN algorithm, RABIN algorithm, and DES (data encryption 
standard) algorithm, see Schneier, B., Applied Cryptography, supra. A person 
skilled in the art will know how to apply such encryption algorithms to a digital 
object 

It is noted that the encryption step, either using a public key system or a 
private key system, can be used to encrypt the digital object directly without 
processing through a hash fimction, as long as the private key is kept secret. 
However, if the digital image is large, extremely long computational time will be 
needed. The hash fimction reduces the size of the data that requires encryption to 
produce the watermark. 

After the encrypted message digest S is formed, it is processed (block 106) to 
derive the watermarked object I^iJ)- In an embodiment, shown in Fig. 2, to derive 
the watemlarked object I'(i j), the encrypted message digest S is modulated, e.g., to 

11 
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modify the amplitude of the signal, to spread over a perceptually significant region of 
the spectrum (block 108). By this process (block 108), a key vector V, representing a 
physical domain signal, is obtained. Spreading the message digest S over a large part 
• of the spectrum has the advantage of rendering the watermark substantially 

5 imperceptible to the human sense organ for which the object is designed. Further, the 

watermark will be preserved if the data is manipulated by processes such as 
compression or cropping. For example, spreading the message digest S over a 
subst£mtiai part of the spectrum would not overly distort a certain small range of 
colors in a visual image or distort a certain small range of audio firequency. Since a 

10 usefixl watermark preferably is preserved when undergoing lossy compression or 

cropping, the watermark is placed in the portion of the spectrum in the perceptually 
significant portion of the spectral firequencies. If such a watermariced object is 
cropped to remove the watermark (e.g., by an attacker), the object would be distorted 
sufficiently that its quality would be substantially inferior to the original object or the 

15 watermarked object. A person skilled in the art will know what portion of the 

spectrum to modulate for a particular object without undue e^qperimentation. 

From the key vector V, a watermark vector W applicable for inserting into a 
selected portion of the original object I(i j) is obtained (block 110). As will be 
described later, the watermark vector W is dependent on the particular portion of the 

20 original object I(i j) selected for the insertion of the watermaric After incorporating 

the watermark into the original object I(ij) by combining the watermark vector V 
with the selected portion of the original object I(ij), the watermariced object r(ij) is 
obtained (block 112). 

12 
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To better illustrate the preferred embodiment of the invention, an example of 
watermaricing a visual image is provided below. It is to be understood that other 
types of physical objects, for example, those mentioned above, e.g., audio digital 
objects, tactile digital object, and the like, can be similarly watermarked by a person 
skilled in the art based on the present disclosure. 

Example 

The process is described in general and illustrated in Fig. 3 as follows 

(1) A digital object image I(iJ) with m rows and n column of pixels is 
obtained. A message digest of the data bits in I(i j) is computed, using a standard 
message algorithm such as the MD5 function, to obtain the message digest M. 

(2) The message digest is signed to create the owner*s signature, with 
encryption methods such as the RS A method or Elliptic Curve, thus obtaining the 
encrypted message digest S. 

(3) Let S be the bits of the signature for constmcting a watermark. For 
example, S may have 512 bits. A vector U of n entries is constructed, where n 
corresponds to the number of columns of pixels. Alternately, die vector can have m 
entries corresponding to the nimiber of rows, m. If that is the case, the following 
steps referring to rows will be ^plied to columns instead, and vice versa. In 
constructing U in this example, the second bit to the 6Sth bits are assigned a 
modulation value depending on whether the corresponding bits (e.g., the fu:st 64) of 
the encrypted message digest S are 0 or 1 . The rest of the bits in encrypted message 
digest S arfe assigned values of 0. The first bit of S is assigned a value of 0 to 
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correspond to the DC component of the pixels. For example, in the second to the 
65th bits of U, a bit is assigned a value of -L for a corresponding bit of 0 in S; the bit 
in U is assigned a 1 for a corresponding bit of 1 in S. It is to be understood that the 
modulation can have other optional values, as long as they are consistent and the 
resulting watermark would not overly distort the object For example, a bit in U, 
instead of having a value of -1, may be assigned a value of 2; and a bit of 1 in U, may 
be assigned a value of -1 . Furthermore, the first bits of U may have more bits or less 
bits than 64. However, a longer U would require more computer power to implement 
the watermark, and a smaller U is more prompt to be broken by an attacker. The first 
bits of U correspond to the lower firequencies, which are the more conceptually 
significant frequencies of the visual image object This is also true for audio digital 
works. However, it is conceivable that in other works where the perceptually 
significant bits are in the higher firequencies, the higher frequency bits of U will be 
modulated. Furthermore, the first bits of U need not be based on the first bits of S, 
but optionally can be based on some other bits as long as they are consistent For 
example, the first bits of U can be based on the last bits, the middle bits, or alternate 
bits, or the like of S. The first bit in S (corresponding to the DC component) having a 
value of 0 nudces the watermaric resistant to changes in brightness or contrast 

(4) A reverse Fourier transform of U is performed to obtain a key vector V. It 
is to be understood that other types of transforms, e.g., reverse Discrete Cosine 
Transform (DCT), can be used to transform U from the frequency domain to a 
physical domain, e.g. time domain (for example, for audio and other time variant 
signals) or' spatial domain (for, example, for images, video, or other spatially varying 
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signals). A person skilled in the art will know how to select and apply such 
transforms to obtain the key vector V based.on the present disclosure to derive a 
watermark. 

(5) A portion of the original object I(i j) is selected, e.g., b contiguous rows, 
and this portion is averaged to construct a reference vector A relating to the image 
pixels for orthogonalization calculation. In this example, the reference vector A is an 
average vector calculated by averaging data. Depending on the object, one may want 
to select the whole object, or a portion thereof that has conceptually important details. 
For example, b can be 16 in the middle section of the image, indicating that 16 rows 
in the middle section is to be averaged to form A. Then, to reduce the risk that the 
key vector V would have a dependency on the vector A, i.e., would depend on the 
colunuiic position of the elements in the b rows of the object, preferably the key 
vector V is orthogonalized with respect to the reference vector A, thereby obtain a 
watermark vector W. The orthogonalization can be represented by the equation: 



W = V-( i^ A )A 



Eq. 1 



where V = 



^JV•V 



and A = 
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is the unit vector along A. The watermark vector W represents the frequency and 
magnitude data of the digital watermaric that is to be incorporated into the original 
object I(lj). 

(6) The watermark vector W is incorporated into the original object I(iJ), 
preferably, by inserting into the portion of the I(i J) from which the reference vector 
A has been derived. A common method of inserting the watermark vector W is by 
adding a small scaled version of W back to each of the b rows selected for the 
reference vector A earlier. This method of obtaining the watermarked elements 
I'(iJ) in the object can be represented by the following equation: 



In Eq. 2 a is a proportional constant, which may vary depending on the positions of i 
or j if preferred. 

In a preferred embodiment. 



Typically, c is chosen such that the watermark signal is roughly - 40dB PSNR (peak 
signal to noise ratio). Other methods of inserting watermarks can also be used. For 
example, multiple scaling factors a that depend on other factors can be used. 
Additional'watermarks can be added to other locations of the digital object as desired. 



r(ij)= I(iJ) + a(ij)W(ij) 



Eq.2 



I(i j)= c cos ( 27ri / b) 



Eq,3. 
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by repeating the orthogonalization and the watermarking steps using the same key 
vector V. Alternative methods of incorporaLting watermark using watermark factors 
are apparent to one skilled in the art based on the present disclosure (see, e.g.. Cox, et 
aL supra, which is incorporated by reference herein). In the example shown in Fig. 3, 
the b rows of the original digital object are replaced with the b rows of watermarked 
pixels. Thus the watermarked object r(ij) (which has the same number of rows m 
and the same number of colunms, n, as the original object) includes b rows of 
watermarked elements, whereas the remaining (m-b) rows of elements remain 
unchanged from the same rows in the original digital object The resulting watermark 
is a one-dimensional watermark in that it involves all the colunms in the variation of i 
or j in Eqs. 2 to 3, or other similar equations. This one-dimensional watermaric is 
mathematically simpler than a two dimensional watenooaric in which the W varies as a 
fimction of the rows as well as the colunains. However, if desired, a two-dimensional 
watermaric can be created such that a watermark matrix W„ having elements that are 
derived from the process of one-way hash function and encryption. For example, W„ 
can be derived from V by incorporating V with various proportional constants into 
the elements at various columns and rows in W„. This foregoing description is just 
one example, a skilled person will know how to derive a two-dimensional watermark 
from the present disclosure. 

Given a suspect image I"(i j) suspected of being derived (e.g., copied) from 
the watermarked object I^i J)> suspect image can be evaluated by the following 
method, which is illustrated in Fig. 4. This method requires the watermaric vector V 
and the average row vector A. In this method, each block of b contiguous rows in 
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the suspect image for the watermark is successively evaluated. For example, the first 
b contiguous rows can be evaluated, then tl^e b contiguous rows starting from the 
second row to the (l+b)th row, then the third row to the (2+b)th row, and so on. 

(A) In each block to be evaluated, the b rows are averaged to obtain a 
reference vector A". 

(B) The vector A" is orthogonalized with respect to A to obtain a suspect 
watermark vector X. The mathematics of orthogonalizing A" is similar to that of 
orthogonalizing the key vector V in Eq. 1 . 

(C) The relative closeness of X to W is computed, e.g., by calculating the 
correlation between the W and X. The equation for calculating the conrelation 
between X and W a is as follows: 



Correlation = ■ , Eq. 4 



(D) Step (C) is repeated to compute the relative closeness of X to W for all the 
blocks of contiguous b rows. The maximum of the relative closeness over all the 
blocks of b rows is then taken. If this relative closeness is above a predetermined 
threshold value, then the suspect image is deemed to contain the watermaric. 



18 



wo 99/17537 



PCT/US98/20196 



(E) If the suspect image r*(i j) is cropped or scaled, a horizontal search by 
searching at various match locations along ^e reference vector A" can be performed 
to get the best relative closeness, i.e., to arrive at the maximum correlation of the 
watermaric to the suspect object. Further, a search can be conducted for reflections of 
the digital object about the axes. 

(F) Synthesize a number of random candidate watermark vectors (e.g., one 
hundred watermarks) with the same spectral properties as V. Compute the 
correlation of each of these candidate vectors with the suspect digital object I"(iJ), at 
the location, scale, and crop factors at which the correlation of W was maximized. 

(G) Compare the correlation obtained from the original watermark against the 
correlation obtained for the random vectors. If the former and the latter are far apart, 
then it is likely that the suspect object I"(ij) contains the watermark key vector V. In 
other words, it is likely that suspect object I"(i j) has the watermark of the original 
object I(i j), and therefore is likely to have been derived from it 

The above deals with detecting whether a suspect object is derived from an 
original object In the case that the ownership of a digital object J(i j), e.g., a digital 
image, is in dispute, the ownership can be established via determining the presence of 
a watermaric by a neutral third party such as a judge. A person clainxing ownership of 
the digital object will present to the judge his original image I(i j) corresponding to J(i j) 
and the signature S of the hash of that original digital object, and his public key for 
decryption. He would further declare the location in the digital object J(iJ) where the 
watermark can be found and the scaling and cropping factors of J(i j) with respect to his 
original object I(ij). The judge can verify using the following process. 

19 



wo 99/17537 



PCT/US98/20196 



(A) Compute a message digest of the bits in I(i j). Decrypt S with the public key 
presented by the claimant. The bit strings.for the message digest computed and the 
decrypted S should be the same if the claimant is the owner of J(i j). Otherwise, reject 
the claimant's claim of ownership. 

(B) Construct the watermark vector V from the signature S. At the specified 
location of the object J(i j) compute the correlation of the watermaric vector V after 
compensating for cropping and scaling. Use the original object I(iJ) to compute the 
corresponding reference vector A. 

(C) Synthesize random candidate watermarks with the same spectral properties 
of V. Compute the correlation of each of these candidate vectors with die object J(iJ) 
and compare the correlation obtained with V to the correlation obtained with the random 
candidate vectors as in the aforementioned method for dectecting the presence of a 
watermark in a suspect digital object If the two correlation types are far apart, then it 
is likely the object J(i j) contains the watermark vector V, i.e., contains the watermark 
of the object I(i j) provided by the claimant 

In this example a digital image was evaluated. The original image object used in 
this illustrative example is shown in the picture of Fig. 5. The original image object has 
256 (m rows) x 384 (n columns) of pixels. The watermarked image is shown in Fig. 6. 
The strength of the watermaric is -45DBPSNIL A comparison ofFig. 5 and Fig. 6 shows 
that there is no perceptual difference that is distinguishable to the human eye. What can 
be distinguished by the human sense organs, e.g., the eye, has been known in the art or 
can be determined without undue experimentation. Fig. 7 shows the correlation spread 
obtained by a verification algorithm for the watermariced image ofFig. 6 using the above 
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correlation method. In Fig. 7, the abscissa shows the various spurious watermarks with 
the same spectral properties as the true watermark embedded in the image. The spurious 
watermarks were synthesized by the verification algorithm. The spike "sp" in the middle 
of the figure corresponds to the correlation when applied to the image with the true 
watermark, whereas the rest of the graph shows the correlation when applied to 
randomly generated watermarks and the true watermark of the same spectral bandwidth. 

To evaluate the robustness of the watermark in its resistance to distortion by 
cropping and compression, the digital object of Fig. 5 was cropped to 176 X 274 pixels 
and JPEG.compressed with substantial loss, achieving a compression ration of 28.6 
(JPEG is a standard lossy compression method, see Bhaskaran and Konstantinides, 
Image and Video compression standards^ Kluwer Publishers). The cropped, compressed 
image was decompressed to result in the image of Fig. 8. Fig. 9 shows the correlation 
spread for the image of Fig. 8. A tall spike "spc" is clearly seen in the middle of the 
spread, indicating the presence of the watermark, thus identifying that the picture of Fig. 
9 as being derived from the image of Fig. 5. Similarly, making a downscaled version of 
the image of Fig. 5 and computing the correlation spread of the downscaled image (not 
shown in the figures) indicate that the watermarking was verifiable after downscaling, 
thus showing that the watermark is resistant to scaling. Further, Fig. 10 shows the 
picture of the image ofFig. 5 after truncation with setting to zero the five least significan 
bits of each eight-bit pixel in the watermariced image of Fig. 6. Fig. 1 1 shows the 
correlation spread obtained by processing through the detection procedure for the image 
of Fig. 10. Again, a clear spike "spt" is seen in the correlation spread. Thus, this 
evaluation on truncation shows that the watermark is resistant to truncation. 
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The technique of watermarking a digital object and evaluating a suspect digital 
object for the presence of the watermark according to the present invention can be 
implemented with digital electronics that are capable of data manipulation and 
calculation based on the equations described herein above. Such applicable digital 
electronics include microprocessors and computers, e.g., personal computers, 
minicomputers, and mainframe computers. Furthermore, the algorithms for the data 
manipulation and calculation can be stored in digital storage devices, such as compact 
discs, floppy discs, hard discs, magnetic tapes, and the like, which can then be loaded or 
read into the microprocessors or computers for implementation of the watermarking and 
evacuation processes. Such digital storage devices are generally articles of manufacture 
having a suitable digital storage medium readable by the microprocessors or computers. 
It is also contemplated that various computers can be networiced so that digital objects 
can be transferred between computers to be watermariced and evaluated for watermarics. 
It is also to be understood that the various steps in the watermarking process described 
above can be done separately by different computers and processors and the results be 
combined to achieve the overall function of watermaricing or evaluation for the presence 
of watermaric, as well as both. 

Althougji the preferred embodiment of the present invention has been described 
and illustrated in detail, it is to be understood tfiat a person skilled in the art can make 
modifications within the scope of the invention. 
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WHAT IS CLAIMED IS: 

L A method for identifying digital object using digital watermark, 

comprising: 

(a) encrypting a message derived from source data on the digital 
object to obtain an encrypted message digest (S); and 

(b) deriving a watermark from the encrypted message digest (S) and 
incorporating into the source data. 

2. A method according to claun 1 wherein the message is obtained via 
performing a hash fiinction on the source data on the digital object to obtain a message 
digest (M) on the digital object, and wherein the message digest (M) is the message 
encrypted with the signature encryption key to obtain the encrypted message digest (S). 

3. A method according to claim I wherein the watermaric is a physical 
domain watermark and the method further comprises incorporating the physical domain 
watermark to at least a portion of the source data. 

4. A method according to claim 3 further comprising transforming a 
frequency domain vector (U) derived from the encrypted message digest (S) to physical 
domain in deriving the watermark. 
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5. A method according to claim 4 further comprising deriving the 
frequency domain vector (U) by modulating-at least a portion of the encrypted message 
digest (S) to obtain at least a portion of the vector (U). 

6. A method according to claim 5 wherein a portion of the vector (U) 
corresponds to low frequencies and another portion of (U) corresponds to high 
frequencies, the portion of U corresponding to low frequencies being derived by 
modulating at least a portion of the encrypted message digest (S). 

7. A method according to claim 6 wherein the portion of the vector (U) 
conresponding to low frequencies are modulated to have more significant impact on 
amplitude of the watermark than the portion of the vector (U) corresponding to high 
fi^uencies. 

8 . A method according to claim 7 wherein the portion of the vector (U) 
corresponding to low frequency has negative value in elements corresponding to "0" bits 
of the at least a portion of the encrypted message digest (S) and has positive value in 
elements corresponding to "1" bits of the at least a portion of the encrypted message 
digest (S); and \^erein the portion of (U) corresponding to hi^ frequencies have 
elements of zero value. 

9. A method according to claim 1 wherein the source data consist of 
rows and cblunms of pixels and the watermark is represented by a watermai^ vector (W) 
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having a dimension corresponding to the number of rows (m) or the number of columns 
(n) of the pixels. 

10. A method according to claim 9 wherein a pixel contains data on a 
discrete section of an image object 

11. A method according to claim 9 wherein a pixel contains data on a 
discrete section of audio object 

1 2. A method according to claim 9 wherein the watermark incorporated 
into the source data is orthogonal to the data to which the watermark is added. 

13. A mediod according to claim 9 further comprising deriving from 
the source data a source data vector (A) having the same dimension as that of the 
watermark vector (W) by selecting at least a portion of the source data and further 
comprismg deriving the watermark vector (W) based on the encrypted message digest 
(S) such that watermaric vector (W) is orthogonal to source data vector (A); and further 
comprising combining die watermaric vector (W) with the data in the selected portion of 
the source data from which source data vector (A) is derived to form watermarked data. 

14. A method according to claim 9 further comprising comparing the 
at least a portion of the source data before incorporation of the watermark to after 
incorporation of the watermark. 
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15. A method according to claim 14 further comprising finding the 
correlation between the watermark vector (W) and a target vector (X) derived from data 
suspected of containing the watermark, wherein said target vector (X) is orthogonal to 
the source data to which the watermark is incorporated. 

16. A method for identifying data using digital watermark, comprising: 

(a) performing a one-way function on source data to obtain a 
message digest (M); 

(b) encrypting the message digest (M) with a signature encryption 
key to obtain an encrypted message digest (S); 

(c) deriving a frequency domain vector (U) from the encrypted 
message digest (S) by modulating a portion of the encrypted message digest (S) 
corresponding to low frequencies more than a portion corresponding to hig^ 
frequencies; 

(d) transforming the frequency domain vector (U) into a physical 
domain key vector (V); 

(e) selecting a portion of the source data and deriving a 
watermarking vector ( W) from the frequency domain vector (U) orthogonal to the 
selected source data; and 

(f) combining the selected source data wi A the watermarking vector 
(W) in the physical domain. 



17.* 



A system for identifying data using digital watermaric, comprising: 
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(a) means for encrypting a message derived from source data with 
a signature encryption key to obtains encrypted message digest (S); and 

(b) means for deriving a watermark from the encrypted message 
digest (S) and incorporating into the source data, 

18. A system according to claim 17 further comprising a means for 
performing a hash function on the source data to obtain a message digest (M) and 
wherein the means for encrypting encrypts the message digest (M) with the signature 
encryption key to obtain the encrypted message digest (S). 

19. A system according to claim 17 wherein the water mark is a 
physical domain watermark and the means for deriving incorporates the physical domain 
watermaric to at least a portion of the source data. 

20. A system according to claim 19 wherein the means for deriving 
derives a frequency domain vector (U) from the encrypted message digest (S) and 
transforms the vector (U) to physical domain in deriving the watermark. 

2L A system according to claim 20 wherein the means for deriving 

derives the frequency domain vector (U) by modulating at least a portion of the 
encrypted message digest (S) to obtain at least a portion of the vector (U). 
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22. A system according to claim 21 wherein the means for deriving 
manages the source data as rows and columns of pixels and derives a watermark vector 
(W) based on the vector (U), the watermark vector (W) having a dimension 
corresponding to the number of rows (m) or the number of columns (n) of the pixels. 

23. A system according to claim 22 wherein the means for deriving 
derives from the source data a source data vector (A) having the same dimension as that 
of the watermark vector (W) by selecting at least a portion of the source data and 
wherein the watermark vector (W) is orthogonal to soxirce data vector (A). 

24. A system according to claim 23 fiirther comprising a means for 
comparing a set of target data with the source data, the means for comparing compares 
a target vector (X) derived from the target data to the source data, the target vector (X) 
being orthogonal to the source data vector (A). 

25. An article of manufacture comprising a program storage medium, 
tangibly embodying a program code means readable by a computer to cause the 
computer to identifying a digital object using a digital watermark, comprisuig: 

(a) code means for performing a one-way frmction on source data 
on the digital object to obtain a message digest (M) on the source data; 

(b) code means for encrypting the message digest (M) with a 
signature encryption key to obtain an encrypted message digest (S); 
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(c) code means for deriving a watermark from the encrypted 
message digest (S) via a transforming a portion of the message digest (S) as 
frequency domain into a physical domain before resulting in a one-dimensional 
wateraiark for incorporating uxto the source data; and 

(d) code means for incorporating the one dimensional watermark 
into the source data. 
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